User-Centred Requirements Handbook |
This section provides guidance in testing the feasibility of the system concept and to generate possible changes to it. The assessment may also help in selecting the most suitable design concept from a number of competing concepts.
Process
Controlled experiments can be used to test the usability of a prototype or of a simulation of the future system concept. Representative users are asked to perform tasks with the prototype to help check whether the system concept will be an acceptable basis for the new system. Specific elements of the system design may be tested as required; for example input devices and icon sets might be examined.
The results will also help in generating usability goals for the future system when it is developed fully.
For more information about controlled testing see Part C, section 4.2.
The basic method is as follows:
1. Define the sample of users (number in sample and their characteristics) that will perform the tasks. To do this refer to the list of users (section 1.2) and the user characteristics (section 1.3).
2. Define the list of tasks that they will carry out, referring to the task scenarios developed in section 2.3.
3. Define the conditions that will be set up for running the trials. To identify relevant conditions, refer to characteristics of the physical environment (section 1.5), organisational environment (section 1.6) and technical environment (section 1.4).
4. Define suitable measures to assess each task performed with the system e.g. task achievement, task time, satisfaction or ease of use ratings, and how data will be gathered to take the measurements.
5. Define how much introduction will be given to the users before the trials and how help will be offered during the trials. Record these in a table similar to Form 3.11.1 below. When an operational version of the system is ready, the user trials will be run using it.
6. Run the user trials capturing performance measures, rating scale scores, and user comments about the system for each task. These results will be recorded in a table similar to Form 3.11.2 below.
7. Following the user test, the results will be reviewed and the system design will be modified to address any problems identified. The results will also be used as a baseline for testing the system when it is implemented.
Form 3.11.1 - Usability test plan (Example)